LinXGBoost: Extension of XGBoost to Generalized Local Linear Models
نویسنده
چکیده
XGBoost is often presented as the algorithm that wins every ML competition. Surprisingly, this is true even though predictions are piecewise constant. This might be justified in high dimensional input spaces, but when the number of features is low, a piecewise linear model is likely to perform better. XGBoost was extended into LinXGBoost that stores at each leaf a linear model. This extension, equivalent to piecewise regularized least-squares, is particularly attractive for regression of functions that exhibits jumps or discontinuities. Those functions are notoriously hard to regress. Our extension is compared to the vanilla XGBoost and Random Forest in experiments on both synthetic and real-world data sets.
منابع مشابه
Extension functors of generalized local cohomology modules and Serre subcategories
In this paper we present several results concerning the cofiniteness of generalized local cohomology modules.
متن کاملA Local Estimating Approach in Multi-categorical Varying-coeecient Models
Varying-coeecient models are an extension of generalized linear models. In this case the coeecients are allowed to vary smoothly with the value of other variables. We present a "local Fisher scoring backktting" algorithm based on a weighted local likelihood approach where the components are iteratively tted. An example will illustrate the usefulness of this approach.
متن کاملDMGroup at EmoInt-2017: Emotion Intensity Using Ensemble Method
In this paper, we present a novel ensemble learning architecture for emotion intensity analysis, particularly a novel framework of ensemble method. The ensemble method has two stages and each stage includes several single machine learning models. In stage1, we employ both linear and nonlinear regression models to obtain a more diverse emotion intensity representation. In stage2, we use two regr...
متن کاملThe Negative Binomial Distribution Efficiency in Finite Mixture of Semi-parametric Generalized Linear Models
Introduction Selection the appropriate statistical model for the response variable is one of the most important problem in the finite mixture of generalized linear models. One of the distributions which it has a problem in a finite mixture of semi-parametric generalized statistical models, is the Poisson distribution. In this paper, to overcome over dispersion and computational burden, finite ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1710.03634 شماره
صفحات -
تاریخ انتشار 2017